Search engines provide a powerful tool for locating content in documents in a large database of documents, such as the documents on the Internet or World Wide Web (WWW), or the documents stored on the computers of an Intranet. The documents are located by searching an index of documents in response to a search query submitted by a user. The query has one or more words, terms, keywords and/or phrases. The document index is generated by scanning the documents using one or more network crawlers (or web crawlers). When the number of documents to be indexed is large (e.g., billions of documents), accomplishing such scanning in a timely manner usually involves multiple crawlers operating in parallel.
During the scanning of documents by one or more crawlers, additional content or documents may be discovered based on links to such additional content or documents embedded in the documents that are scanned. One existing approach to providing links to additional content or documents is in the form of anchor tags. In hypertext documents, anchor tags may include links to other documents or to other parts of the same document. The existing anchor tags, however, have several limitations. Notably, the information in the anchor tags only convey content or document locations. The anchor tags do not convey opinions about the content or documents referenced by the anchor tags. In general, anchor tags also have not been used to convey weighting of a relative importance of the locations referenced by the anchor tags. And the information in existing anchor tags is public. There is no mechanism to secure the information in an anchor tag such that it may only be viewed by a restricted audience. There is a need, therefore, for improved anchor tags for use by search engines.